1. Field of the Invention
The present invention relates to an image processing method and apparatus and a computer-readable storage medium. More particularly, the present invention relates to an image processing method and an image processing apparatus that correct image distortions caused by oblique imaging in which an original image on the object plane is taken from an oblique direction. Further, the present invention relates to a computer-readable storage medium that causes a computer to perform the image distortion correction processing to correct image distortions caused by the oblique imaging.
2. Description of the Related Art
With a rapid progress in the development of recent computer networks, there is an increasing demand for quickly acquiring important information in all aspects of computer systems. Attention is being given to portable image input devices that can easily and accurately capture an image of a document or a photograph and transfer it into the storage of a computer system. As the practical application, an image input device, such as a digital still camera, is used to capture a document or photographic image by taking the original image on the object plane (or the paper).
However, when the original image on the object plane is taken by the image input device from an oblique direction with the imaging plane of the image input device being not parallel to the object plane, image distortions may take place due to the oblique imaging. The readability of the captured information will be deteriorated, and it is desired to provide means for correcting such image distortions. Hereinafter, the image distortions of this kind will be called the oblique-type image distortions.
Several image processing techniques for correcting the oblique-type image distortions to increase the readability of the captured information are known. First, one major approach is that, when capturing an original image of an object having an A4 or larger size (such as a poster), the image distortions due to oblique imaging are corrected in order to increase the readability of the captured information. For this purpose, it is required that the object image to be corrected contains a desired region of the object. Namely, it is necessary to perform the capturing of the original image such that at least one of a plurality of object images which are taken by the user from the object contains the desired region of the object. It is desirable to provide the user with means for selecting one of the plurality of object images that is a standard image on which the correction of the image distortions due to the oblique imaging is based. In addition, it is desirable to provide means for notifying the user that the standard image is currently taken as one of the object images when capturing the object images.
Secondly, it is demanded that images of large-size objects, such as a newspaper, having a size larger than A4 size, be captured with a high resolution by using a portable image input device or image processing apparatus. For this purpose, an image processing technique is proposed in which a composite image is created with a certain high resolution by combining together a plurality of partially overlapping images which are taken by using an image processing apparatus. In cases in which the object is assumed to be planar, the image processing technique applies geometrical transformation formulas, such as affine transformation or perspective affinity, for the combination of a plurality of partially overlapping images so as to create a composite image. The technique is summarized in “Computer Vision—Technological Comments and Future Trends” by T. Matsuyama et al., published in June 1998 by New Technology Communications Co., Ltd. in Japan.
The basic concept of the above technique is that the respective split images are transformed into partial object images on a standard image surface, and they are combined together based on the standard image to form a composite image. However, there is a problem that if the partial object images on the standard image surface contain the oblique-type image distortions, the resulting composite image also contains the image distortions. In other words, the magnitude of the image distortions in the composite image is varied depending on the standard image which is selected from among the split images and used to create the composite image from the split images. The problem of the conventional image processing method will now be described with reference to FIG. 1.
As indicated in FIG. 1(a), it is supposed that the object plane PL is imaged from three different directions D1, D2 and D3, and the respective split images IM1, IM2 and IM3 are obtained. When the split image IM1 is selected as the standard image, a composite image IMA is created from the split images based on the standard image IM1 as indicated in FIG. 1(b). In this case, the original image on the object plane PL is taken from the left-handed oblique direction D1. When the split image IM2 is selected as the standard image, a composite image IMB is created from the split images based on the standard image IM2 as indicated in FIG. 1(c). In this case, the original image on the object plane PL is taken from the substantially front direction D2. As shown in FIGS. 1(b) and (c), the magnitude of the image distortions in the composite image IMA is quite different from the magnitude of the image distortions in the composite image IMB.
As is apparent from the foregoing, in order to effectively correct the image distortions, it is desirable to provide the user of the image processing apparatus with a means for selecting an appropriate standard image having the substantially front direction from among the split images. Further, it is desirable to provide the user of the image processing apparatus with a means for notifying, when taking one of the split images from the original image, the user that a selected one of the split images is currently taken as the standard image.